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Abstract 

In this study, data collected from 875 college freshman and sophomore students enrolled in a 
4-year university in central China are used to examine the applicability and validity of a 
Chinese version of Holland’s Self-Directed Search (SDS) that was adapted in the 1990s. The 
total sample was randomly divided into two groups. Data from the first half of the sample (ni 
= 438) were used to analyze the reliability and validity of the adapted SDS, and 52 items 
were identified for removal due to lack of cross-cultural fit, lack of validity to measure the 
intended construct, and/or being obsolete due to progress in societal and occupational trends. 
The SDS before and after item removal were further compared using confirmatory factor 
analysis with the second half of the sample (n 2 = 437). Practical implications of this empirical 
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modification approach are discussed. This study contributes to the literature by extending the 
current discussion about cross-cultural adaptation of Holland’s inventories and by suggesting 
a methodological approach to improve the reliability and validity of adapted assessment 
tools. 

Keywords: Holland’s theory, Self-Directed Search (SDS), Cross-cultural applicability, 
Assessment tool modification. Vocational counseling in China 

1, Introduction 

Helping students make career decisions is one of the critical functions of higher education 
institutions. In recent decades, there has been an expanding interest in China in Holland’s 
theory and assessment inventories as many secondary and postsecondary education 
institutions started to search for tools to help students’ career planning. In fact, the number of 
publications that offer anecdotal or empirical discussion of using Holland’s theories and 
inventories to guide the career choices of Chinese youths has been on the rise (e.g.. Long, 
Adams, & Tracey, 2005; Long & Peng, 2000; Long, Peng, & Zheng, 1996; Tang, 2001; Tien, 
2009; Zhang, Kong, Chen, & Li, 2014). Growing with the international recognition of 
Holland’s theory and career interest assessment are also serious concerns. First are the 
challenges in adapting measures and assessment tools across culture. Cross-cultural 
applications run the risk that there may be substantive differences in the underlying meaning 
ascribed by each culture to the methods and assessment items, leading to differences in the 
validity and structural properties of the measures (Rounds & Tracey, 1996; Tinsley, 1992). 

Second, the lapse of time adds another complication. Holland’s assessment tools were 
initially developed over half century ago. Since then, rapid development in science and 
technology has transformed the educational and occupational systems (Bullock et ah, 2010; 
Tien, 2010). Even though there have been updates to Holland’s assessment inventories, 
continuing examination is necessary to validate the assessment items, especially on the 
inventories that have been in place for decades. Within this context, this study uses a sample 
of college students from a university in mainland China to examine an adapted version of 
Holland’s Self-Directed Search (SDS, 1994), and proposes empirical-based modifications to 
improve its construct validity while maintaining faithful confirmation to the theoretical 
underpinning of the instrument. 

2, Review of Literature 

2.1 Holland’s Theories and Typology 

The core to Holland’s theory’s is the categorization of six personality types: Realistic, 
Investigative, Artistic, Social, Enterprising, and Conventional (RIASEC), and each type is 
characterized by a constellation of characteristics, interests, beliefs, abilities, values, and 
preferred activities (Nauta, 2010). Holland (1973, 1997) illustrated the structural relations 
among the types in a circular structure of R-I-A-S-E-C (the hexagonal model). The types that 
are more proximate in the hexagon are supposed to be more similar and share stronger 
psychological resemblance than types that are more distant (Holland, 1997; Nauta, 2010; 
Rounds & Tracey, 1996). During the last few decades, Holland’s theory and model have 
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contributed in innumerable and significant ways to generating critical knowledge related to 
eareer development, assessment, and practice (Nauta, 2010; Rounds & Tracey, 1996). 

The infiuenee of Holland’s model grew even stronger when he and colleagues elassified 
occupations using the RIASEC types (Gottfredson & Holland, 1996; Holland, 1975, 1978), 
whieh permit individuals to explore academie majors, oeeupations, and careers that are 
congruent with their personalities (Nauta, 2010). By elassifying both personality types and 
the environments, Holland’s theory highlights the importance of one’s personality being 
congruent with his or her environment. To date, Holland’s RIASEC typology may be the 
most dominant conceptual model in vocational interest assessment. Eor instanee, in addition 
to the assessment scales initially proposed by Holland himself (e.g., Voeational Preference 
Inventory [VPI], Holland, 1985; the Self-Direeted Seareh [SDS], Holland, Eritzsehe, & 
Powell, 1994; and the Position Classification Inventory [PCI], Gottfredson & Holland, 1991), 
there are other well-known measures, sueh as Strong Interest Inventory (Hansen & Campbell, 
1985), Career Decision-Making Interest Survey (Harrington & O’Shea, 1993), and Career 
Assessment Inventory (Johansson, 1986), that were construeted based on Holland’s RIASEC 
model of personalities. Holland’s typology is also the theoretical backbone of major 
vocational assessment programs sueh as the American College Testing Program (1988) and 
the United States Employment Service (U.S. Department of Eabor, 1979). Aecording to Ruff, 
Reardon, and Bertoeh (2008), Holland’s theory and models have appeared ini97 different 
journals in countries around the world and Holland’s RIASEC typology and related 
applications have been cited over 1,600 times in the literature from 1953 to early 2007 
(Bullock et ah, 2010). 

2.2 Cross-Cultural Applicability of Holland’s RIASEC Typology 

Given the popularity and widespread infiuenee of Holland’s theory, model, and assessment 
inventories, it is not surprising that researchers and practitioners outside the U.S. have 
worked to adapt his RIASEC model and related assessment tools with the anticipation that 
they can be useful in investigating vocational interests in different populations (Bulloek et ah, 
2010; Rounds & Traeey, 1996). For instance, Holland’s Self-Directed Seareh (SDS) has been 
translated into more than 30 languages (Bullock et ah, 2010). Most of Holland’s inventories 
have been adapted to studying young people’s occupational interests in other countries, such 
as SDS in Croatia (Sverko & Babarovic, 2006) and in Hong Kong (Eeung & Hou, 2005), 
Personal Globe Inventory (PGI) in Japan (Tracey, 2002), Voeational Preference Inventory 
(VPI) in India (Leong, Austin, Sekaran, & Komarraju, 1998), and Strong Interest Inventory 
(SII) in Korea (Tak, 2004), just to name a few examples. 

Nonetheless, empirieal researeh has yielded inconsistent findings regarding the 
generalizability of Holland’s model and inventories (Bulloek et ah, 2010; Eong et ah, 2005). 
On the one hand, some researchers (e.g., Hansen, 1987; Holland, 1985; Yu & Alvi, 1996) 
provided evidenee that Holland’s model captures the strueture of interests of international and 
cross-eultural populations almost as well as it does for White western test-takers (Bulloek et 
ah, 2010). On the other hand. Rounds and Tracey (1996) used a meta-analysis to show the 
lack of consistent support to Holland’s structural specification of the RIASEC types, and 
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argued that the meaning, interpretations, and intercorrelations of the six types may have 
substantial variations across cultures. At the minimum, the lack of consistency supports 
Ben-Porath’s (1990) argument that it is important for cross-cultural researchers to investigate 
construct validity by ascertaining whether the interpretations associated with an inventory 
developed in the United States can validly be used in another culture. 

2.3 Holland’s Assessments in China 

Research and applications of Holland’s model and assessment inventories have been growing 
in China (e.g., Chen, 2006; Long et ah, 1996; Long & Peng, 2000; Tang, 2001; Tien, 2009; C. 
S. Wong & P. M. Wong, 2006; Yang, Stokes, & Hui, 2005), and researchers are aware of the 
complications related to cross-cultural fit of the RIASEC model due to drastically different 
cultural and social settings. For instance, Farh et al. (1998) found that the circular RIASEC 
structure did not fit as well to a sample in Hong Kong as it did to American samples. Rounds 
and Tracey (1996) found a similar lack of fit in Taiwanese samples. Yang and colleagues 
(2005) conducted comparisons between two different Chinese samples. They ran 
confirmatory factor analyses to evaluate Holland’s (1973, 1985, 1997) structural hypotheses 
with SDS (Holland, 1994) and suggested that the circumplex model was generally not 
supported for Chinese people. 

A few researchers have tried to localize career interest assessment by creating their own 
inventories based on Holland’s RIASEC coding system (e.g.. Fang et ah, 1996; Tien, 2013; C. 
S. Wong & P. M. Wong, 2006), while others (e.g., Chen, 2006; Long et ah, 1996) took steps 
to alter the original U.S. version of an instrument to address the differences in cultural and 
occupational meanings in items used in Holland’s assessments. In mainland China, one of the 
most popular career interest assessment inventories is the Chinese adaptation of the SDS, 
which was translated and introduced in the early 1990s (Long, 1991) and later modified by 
Long and colleagues (1996) based on the social, cultural, and occupational structure in China 
to better serve the local population. In the modification, the original metric structure of the 
RIASEC model was kept as well as the subscales measuring activities, competence, and 
occupational interests within each of the six types, thus maintaining the eighteen (6 by 3 = 18) 
clusters of items along with the self-estimation ratings at the end. Nonetheless, the number of 
items within each cluster was reduced to 10, and items that were judged not applicable to 
Chinese population were either modified or replaced. For instance, an item on Holland’s 
original SDS, “Fill out income tax forms” was removed because “income tax forms” was an 
unknown term to Chinese back in the 1990. Another example of localization was the 
modification of the original item “I can do painting, watercolor, or sculpture,” into “I can do 
painting, drawing, or calligraphy” to reflect the fact that calligraphy was a common form of 
art in China. 

The popularity of the adapted version of SDS has been increasing with people’s rapidly 
growing interest in career interest assessment. For instance, the adapted SDS has been used in 
a study to establish a system that match the RIASEC personality types with college majors in 
China (Long & Peng, 2000). In another study, the SDS was used to examine college students’ 
occupational choices (Chen, 2006). Recently, the adapted SDS was used to investigate the 
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occupational interests of a new generation of college students (Zhang et al., 2014). As a 
matter of fact, a Google search of “Holland’s SDS” in Chinese S 

yielded roughly 115,000 items, many of which provide free access to the adapted inventory 
and self-directed interpretation of the assessment results. 

3, Research Objective 

The growing literature demonstrates that the influence of the RIASEC model has been 
increasing in many countries, including China. A key issue for cross-culture application of 
assessment inventories is the need to examine the construct validity and generalizability of a 
given instrument (Tinsley, 1992). To investigate construct validity, the first step is to ascertain 
whether the items on a translated instrument from the United States, or on a modified version 
for local application can validly measure the constructs that are intended to be measured 
(Long et ah, 2005). In this study, we focus on the Chinese adaptation of the SDS to illustrate 
issues to be considered in inventory localization. Additionally, we take a step further from 
previous studies on Holland’s cross-cultural applicability to explore an empirical approach 
that improves an inventory by identifying and removing items that appear not to 
appropriately measure the underlying constructs due to either cultural differences or 
obsoleteness. 

4, Methods 

4.1 Participants 

The participants in this study were freshman and sophomore students enrolled in a 4-year 
university in central China. Among a total of 5,933 students (36% of which are females) at 
the two class levels, 1,073 responded to the 180-item survey of Self-Directed Search (SDS), 
which resulted in a final sample of 875 with complete and valid answers. As such, the final 
response rate was at 14.7%. The 875 respondents consisted of 39.5% males and 60.5% 
females and ranged between 16 and 20 years in age. It is obvious that female students had a 
higher likelihood of completing the survey than males, a phenomenon commonly observed in 
survey studies (Porter & Whitcomb, 2005; Sax, Gilmartin, & Bryant, 2003). Distribution of 
respondents was roughly even in STEM (science, technology, engineering, and mathematics, 
n = 456) and non-STEM majors (n = 419). 

4.2 Instrument 

The Chinese version of Holland’s SDS (Long et ah, 1996; Lu, Li, & Tang, 1996) was the 
primary survey instrument used in this study, along with added questions about students’ 
background information (e.g., age, major, and gender). This adapted SDS maintains the 
original structure of the Holland’s RIASEC model, with a total of 180 items measuring 
activity, competence, and occupational interests in each of the six interest categories: 
Realistic (R), Investigative (I), Artistic (A), Social (S), Enterprising (E), and Conventional 
(C). The response format was modified into a Likert scale (i.e., strongly like [5], somewhat 
like [4], indifference [3], somewhat dislike [2], and strongly dislike [1]) in order to improve 
the measurement accuracy (Chen, 2006; Lu et ah, 1996). Eor every respondent, eighteen 
cluster scores were generated (RIASEC types by subscales of activity, competence, and 
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occupational interests) by averaging the responses to individual items belonging to the same 
cluster. Respondents’ self-estimation rating scores were not studied due to its unique data 
structure. There is extensive reliability and validity support for the scales (Nauta, 2010). In 
this study, the coefficient alphas for the six personality types were .93, .89, .91, .87, .90 
and .87, respectively. 

4.3 Data Collection 

Data collection took place in May 2013 and was part of a project titled “Investigation of 
Career Preference and Academic Interest.” Call for response to the SDS survey online was 
initiated by the university’s Provost Office, and conveyed to all freshman and sophomore 
students in 56 majors through college and departmental administrative staff. Students were 
informed about the purpose of the study before a link to the online survey was provided. It 
was made clear that response to the survey was completely voluntary. The summary report 
generated from the SDS would be made available to individual respondents who were 
interested in the evaluation, but no material compensation was offered in any format. 

4.4 Analytical Procedures 

Three steps were taken to fulfill the research objectives of this study. First, the total sample of 
875 college students was randomly divided into two groups. Data from half of the sample {ni 
= 438) were used for exploratory factor analysis (EFA). Specifically, an EFA was done within 
each of the eighteen clusters of items (Table 1) in accordance with the conceptual structure of 
the SDS, using principal component extraction and restricting the number of extracted factors 
to one. The items with the lowest loading on the extracted factor would be considered for 
removal, and the values of Cronbach’s alpha of the cluster before and after the item removal 
were compared. An item would be removed only if the resulting reduction in internal 
reliability, if any, was no greater than .03. After the EEA-guided item removals from the 
subscales, 128 items remained and the SDS was considered as a simplified and more updated 
and culturally-suitable version to measure current Chinese college students’ vocational 
interests. 

Second, both the full and the simplified versions of the adapted SDS were tested with data 
from the second half of the sample (n 2 = 437) using confirmatory factor analysis (CEA). The 
two models were to be compared based on goodness of fit indices including model x > 
RMSEA, SRMR, and CEI. With the goal that only unsuitable items should be removed, the 
simplified version was expected to have better model fit indices than that of the full version; 
particularly, the model x^ should show significant reduction. Einally in the last step, all 875 
respondents were categorized into one of the RIASEC types in accordance with their highest 
score among the six subscales. The categorization with the 180-item SDS and the 
simplifiedl28-item SDS were compared to offer evidence of reliability and concurrent 
validity. 

5, Results 

Descriptive analysis of the data indicated that26.5% of the respondents (232) were 
categorized as S based on their scores from the SDS test, following by 21.7% of A, and 
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17.8% of I. The other three categories, E, C, and R, accounted for approximately 15%, 13%, 
and 6% of the sample, respectively. Significant differences existed between genders (y = 
118.18, p < .001), with girls being more likely in the categories of A, S, and C, and boys 
being significantly more likely in I, E, and S. The gendered pattern of distribution is 
consistent with that of previous studies (Creed et ah, 2009). 

5.1 EFA and Item Identified for Removal 

The 875 respondents were randomly divided into two groups. Using data from the first half 
{n\ = 438), scree plots in the EFAs confirmed that there was only one underlying factor 
within each of the item clusters. As shown in Table 1, in the majority of the subscales, three 
items were identified for removal based on their factor loading; in a few cases, either two or 
four items were removed. The decision to remove an item was based on three guidelines: 1) 
the factor loading of an item was among the lowest in the subscale; 2) there was a perceivable 
gap between the factor loading of the item and the items with relative higher loadings; and 3) 
removing the item would not reduce the Cronbach’s alpha of the cluster over a magnitude 
of .03. Altogether, fifty-two items were removed and the total number of items was reduced 
to 128. With roughly one third reduction in number of items, the Cronbach’s alpha of the 
scales measuring the RIASEC types had little changes from the full version and the values 
were .93, .88, .89, .85, .88 and .87, respectively. 


Table 1. Items identified for Removal (Items with Chinese translation indicate modification 
and/or replacement of items on the original Holland’s SDS inventory) 


Subscale 

Items for Removal (EFA factor loading) 

Internal Reliability 

Before 

After 

Realistic activities (Rl) 

Build things with wood (.545) 

Operate motorized machines or equipment (.356) 

Take a Mechanical drawing course (.596) 

.884 

.898 

Realistic competence 
(R2) 

I can use a multimeter. ^ IM 77 IS 6/] U Tf /fe 
(.670) 

I have operated power tools such as drill press, 
grinder, or sewing machines (.730) 

I can refurbish furniture or woodwork (.666) 

.925 

.921 

Realistic Occupational 

interests (R3) 

Wildlife expert (.220) 

Radio Mechanic (.357) 

Gardener HSllffi (-.002) 

.652 

.721 

Investigative activities 

(11) 

Improve fhiit varieties/cultivate new fruit types, 

ak.S:7k;S:p7j#/±§WffS(l7k;^ (.532) 

Study a scholarly or technical problems (.524) 

.840 

.832 
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Investigative 
competence (12) 

I can name three kinds of food rich in protein, 

(.459) 

I can identify three constellations. 

(.312) 

I can use computer to study a scientific problem 
(.485) 

I attend professional conferences. 
iX (.473) 

.795 

.812 

Investigative 
occupational interests 

(13) 

Anthropologist (.548) 

Mathematician (-527) 

Writer of scientific articles (.446) 

.793 

.768 

Artistic activities (Al) 

Sketch, draw, or paint (.449) 

Read novels or plays (-476) 

Practice calligraphy >] (-543) 

.829 

.826 

Artistic activities (A2) 

I can act in a play (.592) 

I can write a speech (.561) 

.860 

.850 

Artistic occupational 

interests (A3) 

Photographer (Mf^^) (.523) 

Journalist (.558) 

Artist (.554) 

.846 

.834 

Social activities (SI) 

Take care of children JLm) (.449) 

Take field trip with friends 
m (.370) 

Watch or participate in sport games and activities ® 

(.471) 

.804 

.815 

Social competence (S2) 

I am good at explaining things to others (.554) 

I find it easy to talk with all kinds of people (.520) 

Help take care of patients 

(.564) 

.827 

.800 

Social occupational 

interests (S3) 

Rehabilitant counselor (.310) 

Sports coach (.155) 

Tourist guide (-459) 

.662 

.674 

Enterprising activities 
(El) 

Try to persuade others (.495) 

Operate my own service or business 
{All) 

Talk about politics (.583) 

.840 

.831 

Enterprising 
competence (E2) 

I was a leader of a social club or group 
(.587) 

.859 

.836 
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I once was a good leader in student groups 
(.618) 

I am an ambitious and energetic person 

(-621) 



Enterprising 
occupational interests 
(E3) 

TV program producer A (.412) 

Organizer of sport programs (A W 15 3) A A ^ 
(.378) 

Small business owner (.516) 

.766 

.764 

Conventional activities 

(Cl) 

Clean and organize room and desktop 

SfPMl'i] (.529) 

Keep detailed records of expenses (.526) 

Take an information processing course 

(.585) 

.844 

.830 

Conventional 

competence (C2) 

I can type in Chinese fluently A 

(.533) 

I can use a typewriter or copier machine ^T1 AAfT 
(.474) 

I can use an abacus (.309) 

I can enter information at a computer terminal ftlA 
mfM (.450) 

.765 

.773 

Conventional 

occupational interests 

(C3) 

Computer operator (.367) 

Typewriter clerk fTAM (.484) 

.827 

.834 


5.2 CFA and Validation of the Simplified Version of the Inventory 

Both the full (180-item) and the simplified (128-item) versions of the SDS were examined 
with a CFA model using data from the remaining half of the sample {ni = 437). As shown in 
Table 2, the two versions had aceeptable fit indices, but the simplified version had improved 
fit indices by all counts. According to the literature on CFA (Kline, 2010), the guidelines for 
good model fit indices are RMSEA < .08, SRMR < .06, and normed x"< 4, and lower 'i and 
AIC values are always desirable. Not only did the simplified version have lower RMSEA, 
SRMR, and AIC values, it also reduced the model x by 20265.80, which was statistically 
significant at p < .001 with a degree freedom difference of 8056. The CEA results suggest that 
the simplified version of the SDS has a better validity when applied to Chinese college 
students. 
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Table 2. CFA model fit indices to compare the original and simplified versions of the 
inventory 



Model (df) 

Normed y3 

RMSEA(90%CI) 

CFI 

SRMR 

AIC 

Original 

35992.36 (15777) 

2.28 

.054 (.053, .055) 

.92 

.075 

37018.36 

Simplified 

15726.56 (7721) 

2.04 

.049 (.048, .050) 

.93 

.066 

16540.56 


5.3 Consistency in Interest Categorization 

All 875 respondents were categorized into one of the six types in accordance with their 
highest score among the six RIASEC subscales. As shown in Table 3, 81% (n = 707) of them 
had the same categorization on the full and simplified SDS inventories. Three groups on the 
full SDS, R (n = 67), A (n = 123), and E (n = 84), had 67.2%, 77.2%, and 76.2%, partly due 
to their relatively small group sizes, and partly due to the number of respondents shifted to 
adjacent categories. It is worth noting that, 23 students who were the S type by the full SDS 
were now classified as C by the simplified version; and another 23 and 12 were also moved 
from S type to I or E on the simplified version, respectively. These shifts in categorization 
point to the S type as the most unstable categorization based on the 180-item SDS inventory. 


Table 3. Categorization of RIASEC types by original and simplified SDS 


Full 

SDS 

Simplified SDS 

R 

1 

A 

S 

E 

C 

Table Total 

Count 

Row% 

Count 

Row% 

Count 

Row% 

Count 

Row% 

Count 

Row% 

Count 

Row% 

Count 

Col% 

R 

45 

67.2% 

15 

22.4% 



2 

3.0% 

4 

6.0% 

1 

1.5% 

67 

l.TA 

1 

2 

1.6% 

114 

90.5% 

1 

0.8% 

8 

6.3% 



1 

0.8% 

126 

14.4% 

A 

1 

0.8% 

4 

3.3% 

95 

77.2% 

16 

13.0% 

2 

1.6% 

5 

4.1% 

123 

14.1% 

S 

8 

2.1% 

23 

6.0%, 

4 

1.0% 

314 

81.8% 

12 

3.1% 

23 

6.0% 

384 

43.9% 

E 

3 

3.6% 

2 

2.4% 

1 

1.2% 

10 

11.9% 

64 

76.2% 

4 

4.8% 

84 

9.6% 

C 

2 

2.2% 

3 

3.3% 



6 

6.6% 

5 

5.5% 

75 

82.4% 

91 

10.4% 

Total 

61 

7.0% 

161 

18.4% 

101 

11.5% 

356 

40.7% 

87 

9.9% 

109 

12.5% 

875 

100.0% 
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6, Discussion 

The purpose of this study is to improve the validity of an adapted Chinese version of 
Holland’s SDS inventories by empirically identifying and removing items that may not 
properly measure the underlying constructs. Without modifying the metric structure of the 
instrument, the conceptual framework for the RIASEC types remained unchanged. 
Nonetheless, EFA of the measurement items led to removal of 2-4 items from a total of 10 
items in every cluster, and the simplified version of SDS showed better goodness of fit 
indices than the original version of SDS when tested with a different sample of respondents. 

6.1 Items Identified for Removal 

Careful examination of the removed items revealed that the vast majority of items were 
removed for 1) lack of cross-cultural generalizability, 2) lack of validity to measure the 
intended construct, 3) being obsolete due to progress in societal and occupational trends, or 4) 
a combination of the aforementioned three factors. Eor instance, “Build things with wood” 
and “refurbish furniture or woodwork” were handy activities not commonly known to most 
Chinese youth, and “Rehabilitant counselor” was an occupation almost unheard of in Chinese 
culture. They were identified for removal primarily due to lack of cross-cultural 
generalizability. 

During the process of localizing the SDS inventory, the researchers (Eong et ah, 1996; Eu, Ei, 
& Tang, 1996) had used new items to replace those that were believed to be culturally 
unfitted. However, some of the added items may have failed to measure the intended 
construct. For example, “I attend professional conferences” was added to measure 
Investigative competence; unfortunately “attending professional conferences” happened to be 
a concept novel to the majority of students in high schools and colleges. The usability of this 
item for investigating vocational interest is especially questionable given the fact that in 
Chinahigh school students must declare a major prior to college entrance (Tang, 2001). 
Disconnection between the item and the underlying construct also caused the removal of 
items in other places, such as “I once was a good leader in student groups” in Enterprising 
competence and “Wildlife experf ’ in the Realistic occupations. 

Also, the Chinese version of SDS was modified and published in 1996, and many items have 
become outdated during the past two decades. Examples of this are commonly found in items 
measuring Conventional type, including abilities such as “I can enter information at a 
computer terminal” and “I can use an abacus,” and occupations such as “Computer operator” 
and “Typewriter clerk.” 

6.2 Issues in Cross-Cultural Validity 

By identifying and removing the items that no longer properly measuring the underlying 
constructs, the simplified version of the SDS had better validity measures according to the 
CFA results. The simplified version keeps the same RIASEC structure, while using a few 
items to measure the underlying construct. Over 80% of the samples had consistent 
categorization between the 180-item and 128-item versions of SDS. Nonetheless, 
comparisons between the results of the two versions generate questions about identifying the 
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Social type. A comparison of items on Holland’s original SDS and the Chinese version 
revealed that, among the six RIASEC types, the most drastic changes were made to items 
measuring Soeial aetivities, competenee, and occupational interests. The speculation is that 
there are different interpretations of “Social” type in the U.S. and Chinese cultures. As 
reflected in Holland’s SDS, Social activities and occupations are driven primarily by interests 
in providing services to the community (e.g., being teachers, counselors, therapists, and 
volunteers); whereas in China, the word “social” has its focus on interest or activities in 
interacting with a large groups of people. As such, the modified version has items about 
participating in social groups, clubs, and sports events, and interests in being a tourist guide, 
or government employee. Such modifications, even though based on the intent to improve 
cross-eultural applicability, resulted in items that failed to measure the underlying construct. 
For the same reason, the modified and/or replaced items in the Social category may have led 
to inaecurate categorization of individuals by the 180-item version. Once those problematic 
items were identified and removed, substantial changes were observed in the Social 
categorization between the full and the simplified versions of the SDS. 

6.3 Practical Implications 

There are a few things to take away from this study. First, more systematic and well-thought 
steps are needed when localizing or adapting an existing Holland’s inventory into a different 
culture. In this case, the 180-item SDS had great reliability measures as indicated by the high 
Cronbach’s alpha values. Still, many items had less than suflfieient loading to the underlying 
constructs as shown in both the EFA and CFA. The findings of this study suggest that 
examination of the quality of an instrument should go beyond simple reliability measures, 
and it is always necessary to check on additional information about content validity and 
construct validity. Second, statistical analysis as well as input from domain experts can both 
contribute to improving an existing inventory to make it culturally more applicable. The 
initial adaptation of the original Holland’s SDS was based on modifications made by Chinese 
experts’ evaluations and judgments. Even so, well-designed statistical examination of the 
adapted instrument, as illustrated in this study, helps to further elaborate the eonstruct 
grounding of the items. 

Third, monitoring the applieability of existing measurement instruments should be an 
ongoing process, especially in a society that is undergoing rapid cultural and occupational 
changes. As shown in this study, a number of items have already gone obsolete in the 
Chinese version of the SDS adapted two decades ago, a time during which Chinese people 
experienced great economie and cultural progress. Removal or replacement of outdated 
items would improve the validity and continuing applicability of the assessment inventory. 
East but not least, as illustrated by evidenee in this study, fidelity to the grounding theoretical 
framework can be maintained by an assessment tool of fewer items. Relative to a lengthy 
instrument, individuals are more likely to complete a survey of fewer items, and to do so with 
greater care and attention (Deutskens et ah, 2004). This study demonstrated methodological 
steps to simplify a survey instrument while assuring its reliability and validity, which will be 
useful in future research on survey design and improvement. 


226 


www.macrothink.org/jei 




Macrothink 

Institute" 


Journal of Educational Issues 

ISSN 2377-2263 
2016, Vol. 2, No. 1 


6.4 Limitations and Future Directions 

The sample of this study consisted of volunteering freshman and sophomore students from 
one four-year college in China. The sample was not statistically representative to claim 
generalizability of the findings; and we were unable to further evaluate how the sample 
affects construct and predictive validity of the simplified instrument due to limited data 
availability. Thus, further examination of the simplified SDS is needed before applying it to 
empirical research and career counseling practice. Also, the focus of this study is the 
reliability and validity of an adapted SDS. What remains unexamined the internal 
relations among the RIASEC types, represented by a hexagon, that specifies the structural 
relationships of the six different personality types (Holland, 1985, 1997). Given the empirical 
evidence that Holland’s circular relationships differed significantly across countries (Bullock 
et ah, 2010; Farh et ah, 1998), the structural property of the SDS, as measured by the 
simplified version, should be tested in future research. It is also worth noting that inaccurate 
Chinese translation of some of the items may contribute to their misfits and subsequent 
removal in the analyses. If possible, it would be interesting to replicate this study with data 
collected from a comparable sample of American college students using Holland’s original 
SDS, and comparison between the two populations will help better understand issues related 
to cross-cultural applicability. 

7, Conclusion 

Holland’s theory, the RIASEC model, and associated assessment tools help individuals better 
understand their own personality and provide them the guidance in identifying skills, 
occupations, and work environments in which they are more likely to persist and be 
successful (Bullock et ah, 2010). This study contributes to the literature by extending the 
discussion about the cross-cultural adaptation of Holland’s assessment tools and by 
suggesting methodological approaches to improve the reliability and validity of an adapted 
inventory. The findings hold promise for scholars, researchers, and practitioners in vocational 
psychology and guidance in China because theoretically grounded schemes and valid 
assessment tools for identifying vocational personalities and classifying occupations will 
become more useful as China’s economy becomes more urban and market-driven (Bullock et 
ah, 2010). In the long run, we hope this study will contribute to assuring that Holland’s theory, 
model, and assessment tools are appropriately adapted and used in different cultural settings, 
and to increasing the knowledge on how to address the evolving occupational needs of 
individuals. 
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